AP.PUB[AP,SYS] - www.SailDart.org

perm filename AP.PUB[AP,SYS] blob sn#054362 filedate 1973-07-23 generic text, type C, neo UTF8

COMMENT ⊗ VALID 00008 PAGES
C REC PAGE DESCRIPTION
C00001 00001
C00002 00002 .BLANK PAGE
C00009 00003
C00016 00004 .SEC RUNNING APE
C00026 00005 .BEGIN TURN ON "\→" INDENT 0,15 TABS 12,15 GROUP
C00029 00006 .SEC SEARCHING THE NEWS FILE
C00032 00007 .SEC COMMAND FILE INPUT
C00039 00008 .SKIP TO COLUMN 1
C00051 ENDMK
C⊗;
.BLANK PAGE

.MACRO SEC(NAME)⊂
.IF LINES<10 THEN NEXT PAGE;
.GROUP SKIP (IF LINE = 0 THEN 1 ELSE 3);
.ONCE CENTER}NAME
.SKIP;⊃

.TURN ON "{" << this is left on for the whole manuscript >>
.COUNT SPEC << special features on last page(s) >>
.MYDATE←DAY&" "&MONTH[1 TO 3]&" "&YEAR;

.NOFILL
.TURN ON "←→"
STANFORD ARTIFICIAL INTELLIGENCE LABORATORY→{MYDATE}
OPERATING NOTE 72
.GROUP SKIP 9
←READING THE ASSOCIATED PRESS NEWS
.GROUP SKIP 2
←By Martin Frost
.TURN OFF
.SKIP 5
.FILL

ABSTRACT:

We have a line from the Associated Press (AP) over which we
get national and international news (no local news). The line is
read by a program that takes incoming news stories and files them
away on the disk, keeping about 24 hours' worth of news on file at
any given time. This document (which exists as the file APE.ME[S,DOC])
describes usage of programs that allow access to the AP news.

.SKIP 4
The Associated Press news report is made available in these programs
for demonstration and research purposes only and caution must be exercised
to insure that the news is not published or broadcast or publicly displayed
or used for any commercial purpose.

.SKIP 4
This work was supported by the Advanced Research Projects Agency of
the Secretary of Defense under contract SD-183.

.EVERY HEADING({MYDATE},AP News,Page {PAGE})
.SKIP TO COLUMN 1

For use in reading the news, there are two programs on the
system. The first of these is HOT, which is a very small program
that simply types out the stories as they come in.
The second program is called APE; by categorizing each story from
a list of keywords,
it enables the user to selectively read the news on file.

To use the hot line, simply type the monitor command: R HOT.
The program should type back: "...Associated Press news..."; if it
doesn't, then it is having trouble contacting the program [-AP-],
which listens to the AP line. In this case, the program will try for
about thirty seconds to contact [-AP-], after which time it will give up
and tell you so. After "...Associated Press news..." is typed out,
you will get whatever news is coming in. There are times, usually
of only a few minutes duration, when no news is coming in; at such
times, HOT will of course type out nothing. WARNING: Typing control-C or
holding the typeout while the news is coming in will probably cause
HOT to miss some characters. If that happens, your job number will
be scratched from the list of jobs getting the hotline news; so you
will have to restart HOT.

Now before describing the second program (APE), I will
explain a few things about AP news stories. First of all, each story
sent by the AP
has a sequence number which comes at the beginning of the story and a
date and time that come at the end. After the sequence number, we insert
the date and time (Pacific time) we received the story.
The sequence numbers start over
every day, with the first story that comes after about midnight EST getting
number 001. Some special stories (advance stories) are given sequence
numbers out of the normal order; these stories have numbers greater than 400.
The time at the end of each story is the approximate New
York time when the story was sent over the wire.

Every twelve hours (at about noon and midnight EST) there is a
news digest that summarizes the stories that are known to be coming
in over the next twelve hours. The digest at midnight is usually
story number 002 and is called the PMs digest; the one at noon is
usually number 202 and is called the AMs digest. No PMs digest is
sent for Sunday. The digests
are not categorized by the our AP programs;
to access them you must use one of the
two methods described in paragraph {SPEC SPEFORMS} under SPECIAL FEATURES.
Stories that have been mentioned in the latest digest
bear the heading word "BJT" (for "budget").

Each day there are many stories that are corrections or
additions to previous stories. We try to link up such a follow-up
with the original and treat the resultant combination as one story,
although it may be made up of two, three, or even more separately
numbered stories. Any attempt to retrieve with APE any story of
such a group will result in retrieval of all parts of the group
in chronological order. (Long stories are
broken up into smaller parts by the Associated Press; the smaller
parts are called TAKES and each gets its own sequence number. We try
to link all takes of the same story together just like additions and
corrections.)

.SEC "THE ASSOCIATED PRESS EXTRACTOR (APE)"

The program that is used to retrieve news is called APE. It
allows quick access to the stories because of a
data structure that is continually being updated by other programs.
As each story comes in over the AP wire, it is categorized by
keywords from a special list (the keyword dictionary).
For each keyword in the dictionary a list is kept of all the stories that
word occurs in. To access the news, you select the keyword or
combination of keywords that you wish to read about.
A keyword can be either a single word (or number) or a sequence of
words (and/or numbers). For example, the following are some possible
.TURN ON "#"
keywords: WELFARE, WAR, SAN FRANCISCO, UNITED STATES, UNION OF SOVIET
SOCIALIST REPUBLICS, PDP#10, etc.

The keyword dictionary contains about 1000 words,
mostly people's names and names of places (cities, states,
countries). This list is expandable, and if you have any words you
would like added to the list, MAIL a note to ME.
To see a list of the keywords, read the file
WORDS.SRT[AP,SYS].

All input lines to APE should be terminated with carriage returns.
Old APE users should note that this is a change from earlier versions.

.SEC KEYWORD EXPRESSIONS

To retrieve stories using APE, you type in a KEYWORD
EXPRESSION, which may be either a single keyword or an expression
containing keywords and the operators +, -, and *. Each keyword
represents the set of all the stories it occurs in. And the operators
represent the set operations UNION#(+), INTERSECTION#(*), and SET
DIFFERENCE#(-), which are performed on the sets of stories which the
.TURN OFF
keywords represent. Thus, if you want all stories that mention both
Nixon and McGovern, you should type the keyword expression
"NIXON*MCGOVERN". The precedence of operators is the normal one: *
takes precedence over + and -, which have equal precedence.
Operators with equal precedence are evaluated from left to
right. Parentheses may be used freely in keyword expressions. Note
that + - * are BINARY operators only.

To clarify all this a little, here are a few examples:

.BEGIN
.TABS 34,38,69
.TURN ON "%" FOR "#"
.TURN ON "∞\←"
.TURN OFF "-"
←Keyword Expression\\←Meaning\

∞-\\∞-\-

.INDENT ,37
(NIXON-WALLACE+MCGOVERN)*ELECTION\\All stories that mention both
ELECTION and either (1) NIXON and not WALLACE or (2) MCGOVERN
(and possibly WALLACE).

ELECTION-NIXON-WALLACE-MCGOVERN\\All stories that mention
ELECTION but that mention none of NIXON, WALLACE and MCGOVERN.

SAN%FRANCISCO+LOS%ANGELES-WAR\\All stories that mention either
SAN FRANCISCO or LOS ANGELES, but not WAR.
.END

Note: Spaces are needed only to separate individual words of multiple
word keywords, but they may be used anywhere
except in the middle of a word or special form.
.SEC RUNNING APE

To run APE, type the monitor command: R APE. When APE starts
up it reads in various files, and it is possible that another program
will be writing one of these files. In that case, APE will say
"One moment please..." and will wait until it can read the file.
After all the files have been read in, APE will respond with
.BEGIN VERBATIM

KEYWORD EXPRESSION:

.END CONTINUE
You should then type in a keyword expression as defined
in the previous section. APE will count the stories that match your
expression and tell you how many stories it has found, such as:
.BEGIN VERBATIM

5 news item(s) found. Selection:

.END CONTINUE
At this point, you can select any contiguous group from the stories
found. For example, you can read the oldest 4 stories of those
matching your keywords, or you can read the newest 3, or the 2nd
through the 4th, or all of them, or none of them, etc. And you can
have the stories you select typed out, spooled (on the line printer)
and/or saved in a file on your disk area, all by typing in the
appropriate selection line as explained below.

The syntax for the selection line is as follows, where [...] denotes
an optional quantity and ...|... denotes exclusive alternatives:
(The order of different parts of the selection line is irrelevant except
that any filename must come first.)
.BEGIN VERBATIM

[ <filenm> [/Q|/X] ← ] <story selection> [=] [S] [K] [W] [C|F|L|D]

.FILL PREFACE 0; INDENT 0,3;
<filenm> is a filename of up to 6 characters (no extension or PPN is allowed).
If the <filenm> term is present, the stories selected will be saved
in the given file. If you do not say either /Q or /X after the filename,
then if the file already exists, you will be told so and asked what to do.
The presence and end of the <filenm> term is indicated
by the left arrow (←). File output is not allowed with the F or L options
(see below).{BREAK
/Q following the filename means replace file if it already exists.{BREAK
/X following the filename means extend file if it already exists.{SKIP

The <story selection> indicates which stories you wish to select from those
found. It also indicates in what order you want to read the stories. The
syntax for <story selection> is:
.BEGIN VERBATIM

N | <nbr>[:<nbr>] | <empty>

.END CONTINUE
where <nbr> is a positive or negative integer and <empty> is the empty string.
If a single integer (k or -k) appears, it indicates how many stories you
wish to read.
If the number is positive, you will get the k most recent stories; if the
number is negative, you will get the k oldest stories. In either case,
the stories will come out in reverse chronological order, that is, newest
stories first. Two integers
separated by a colon (:) indicate a range of stories. For instance, "2:4"
represents the second through the fourth most recent stories. Negative
numbers in this construction represent the oldest stories; for example,
"-2:-4" represents the second oldest story through the fourth oldest story
in that order. The stories will come out in the order specified; that is,
"-4:-2" represents the same stories as "-2:-4" but in opposite order.
.BREAK; CRBREAK; TURN ON "#"
<empty> means select all the stories in reverse chronological order.
N means select None of the stories.
.SKIP
=#means#reverse the order in which the stories come out.
S#means#Spool the selected stories (not allowed with F or L options).
K#means#Kill automatic reading from command file (see section below on command files).
W#means#type out the Words each story is categorized by.
.SKIP
C#means#Choose which stories get typed out completely (see below).
F#means#type out only the First few lines of each story.
L#means#type out only the Last few lines of each story.
D#means#Dont type out the stories at all (useful if you are saving the stories in a file or spooling them).
.END

.CINT:
If you use the Choose feature, then for each story the first few lines will
be typed out and you will be expected to indicate whether you want
to read the rest of the story. You will NOT be prompted at this point; the
typeout will simply stop, often in the middle of a word.
If you do not want to read the rest of
the story, type just carriage return. To read the rest of the story,
type altmode, linefeed, or any character (except "I") followed by carriage return.
If you don't want to read any more of the stories, type "I" and carriage return.
This has the same effect as [ESC] I followed by carriage return (see paragraph
{SPEC ESCI} under SPECIAL FEATURES).
The character(s) you type will not be echoed, so the story will appear unbroken.
You will be allowed to quit reading a story at the beginning of each
part (take, correction, etc.) of the story. If you are saving stories
in a file or spooling them, then only those you choose to read will be
put in the file and/or spooled.
.BEGIN TURN ON "\→"; INDENT 0,15; TABS 12,15; GROUP

Here are some selection line examples and their meanings.
.SKIP

→2\\Type out the newest two stories.

→-2\\Type out the oldest two stories.
.APART

→=2\\Type out the newest two stories in chronological order.
(Normal order is reverse chronological order.)

\\(Blank line.) Type out all the stories.

→F2\\Type only the first few lines of each of the two newest stories.

→-2:5\\Type out the 2nd oldest story through the 5th newest story.

→=5:-2\\Same as -2:5.

→2:2\\The only way to get just the 2nd newest story.

→=\\Type out all the stories in chronological order.

→FOO←\\Type out all the stories and save them in the file FOO.

→FOO/Q←D\\Dont type out anything, but put all stories into the file FOO.
If the file already exists, then delete the old version.

→FOO/X←SC5\\For the newest 5 stories, type out the first few lines and
let me Choose whether I want to see the rest of the story.
Extend the file FOO with any stories I choose and then Spool it.

→L\\Type out only the last few lines of each story. (The last few lines
include mainly the time and date of the story.)

→N\\Do Nothing with the stories found. (Get next keyword expression.)

------------
.END

If you ask for the stories to be saved in a file, the file will be given
the standard extension ".AP" and will be put on your own disk area (or your
ALIAS area if you currently have an ALIAS). If you ask for
the stories to be spooled but not saved in a file, APE will create a
file with a name like $NEWS0.AP, which will be spooled and then
deleted. (The file $NEWS0.AP will be put on your real disk area
(NOT your ALIAS area) so that the spooler can delete it.)
.SEC SEARCHING THE NEWS FILE

While an expression is being read, if a keyword is encountered
that is not in the keyword dictionary, you will be told so and asked
if you would like a search done for that keyword in the news. If you
want a search done, type "Y" (and a carriage return) for Yes.
Type just carriage return if you dont want a search.

During a search, every time a story is found containing
the searched for keyword, an asterisk (*) will be typed out. Should
you wish to discontinue the search at any time, type a carriage return or,
on Stanford displays, [ESC] I.
Any stories found up to that time will represent the particular keyword
in the expression as if searching had gone to completion. Stories are
searched in the order of newest to oldest. For every
keyword not in the dictionary, a separate search must be done. However,
once you have said Yes to searching, subsequent keywords in the same
expression will be searched for automatically without your being asked.
You may, of course, interrupt such a search (by typing a carriage return
or [ESC] I).

Multiple word keywords may be searched for just like single word keywords,
but only those instances where the whole multiple word keyword occurs
on the same line in the news will be found (this
is the result of an important search optimization).

.TURN ON "#"
Searching the whole news file for a keyword takes about 8 to 10 seconds
of computer time. If, however, an unrecognized keyword occurs as the
SECOND part of an intersection or difference operation (eg, NIXON#*#JJJJ
or NIXON#-#JJJJ), then only the necessary stories are searched and the
search time is generally very much smaller.
.TURN OFF
.SEC COMMAND FILE INPUT

If, in place of a keyword expression, you type an at-sign (@) followed by
a file name (extension and/or PPN allowed!), then APE will endeavor to read
a keyword expression and then possibly a selection line from the file. APE
can handle most (if not all) text file formats, including SOS and E/TV.
After you have opened a command file in this manner, if you type just an
at-sign for a keyword expression, APE will read another keyword expression
(and selection line) from the command file. This can continue until the
end of the file is reached, at which time APE will type out [EOF] to let
you know.

If you follow the at-sign in either case above with an exclamation point (!),
then APE will automatically read from the command file whenever a keyword
expression is needed. This automatic reading from the command file can be
stopped by using the K option in the selection line (see above). This cancels
the effect of the exclamation point. Whenever the selection line is read
from the command file (see below), however, you don't get a chance to type
the K. If you use the system REEnter command or type [ESC] I while stories
are being typed out (see paragraph {SPEC ESCI} under SPECIAL FEATURES), or
if you type I<crlf> when choosing stories (see the Choose option for the selection
line on page {PAGE CINT}), then automatic command file reading will be turned off.
Any error
in an expression read from the file will also turn off automatic reading.

If you type an at-sign (and optionally an exclamation point) without a filename
at a time when you have no command file open, then the standard command file
name APE.CMD will be assumed. You may open the file APE.CMD on someone else's
disk area by typing, for example, "@[FOO,BAZ]" or "@![FOO,BAZ]".

Now a word about how command files are interpreted.

When reading from a command file, APE reads until a semicolon (;)
or comma (,) is found. All carriage returns, linefeeds and form
feeds (page marks) are completely ignored. (That means a keyword
can be split between two lines or even two pages!) If,
when reading a keyword expression from a file, a comma is encountered,
then the stuff following the comma and up to the next comma or semicolon is
assumed to be the selection line you want for this particular keyword expression.
On the other hand, if a keyword expression is terminated with a semicolon,
the selection line will be read from the console instead.
Selection lines in a command file should end with a semicolon. If one ends
with a comma, everything up to the next semicolon will be ignored.

Every keyword expression and selection line read from a command file will be
typed out preceded by an at-sign (@) to indicate that it came from the file.

Finally, whenever an unrecognized keyword is read from a command file,
it is automatically searched for without your being asked. You can, of course,
always interrupt the search (by typing a carriage return or [ESC] I).

Here is a sample command file:

#2+#202,1; TELEVISION+TV; MOVIES,; THEATRE,C; STAGE,STAG/X←C;

This file contains five keyword expressions. The first one will cause the
latest digest (number 2 or 202) to be typed out.
(See paragraph {SPEC SPEFORMS} under SPECIAL FEATURES below.) Next,
if any stories about TELEVISION
or TV are found, the user will be allowed to type in his own selection line.
Then, if any stories about MOVIES
are found, they will automatically be typed out (note the empty selection
line between the comma and the semicolon). If any stories about THEATRE are
found, the user will be allowed to choose which ones he wants. If any
stories about STAGE are found, the user will be allowed to choose which
ones he wants, and those he picks will be added to the file STAG.
.SKIP TO COLUMN 1
.SEC SPECIAL FEATURES
.MACRO SPE(LABEL) ⊂LABEL NEXT SPEC;
{!}.{⊃
.TURN ON "%" FOR "#"

.SPE
%Whenever APE is expecting input, if you type a question mark (?)
and carriage return, you will be given some help regarding what
you are to type in.

.SPE SPEFORMS:
%In addition to normal English keywords, there are two special
forms that can be used as keywords in expressions. The first
consists of a period (.) followed by an unsigned integer, eg., ".18";
if k is the integer following the period, this form represents the
newest k stories that have come in. The second special
form consists of a number sign (#) followed by an unsigned integer,
optionally followed by a colon and another unsigned integer.
The form #k represents all the stories that have k as
their AP sequence number; the form #k:m represents all the stories
with sequence numbers from k to m (wrapping around if k>m).
Using one of these forms is the only way to get the
AP news digests because the digests are not categorized at all.
(Actually, stories #1, #2, #201 and #202 are the ones not
categorized; occassionally the digest has some other sequence number
so it gets categorized.) Here are some examples of keyword expressions
using these special forms.
.BEGIN VERBATIM

CHESS * .10 Among the last 10 stories that
have come in, all those that
mention CHESS.

#2 + #202 All stories with either of these
sequence numbers. (These are the
usual sequence numbers of the
news digests.)

#325:23 All stories with sequence number
greater than or equal to 325 or
less than or equal to 23.
.END

.SPE CURRENTLIST:
%Typing just CARRIAGE RETURN for a keyword expression
(the null keyword expression)
has a special effect; it gives you back the stories
corresponding to the previous keyword expression. These
stories constitute your CURRENT STORY LIST. With this feature you
can get back a second time the stories you just looked at. In fact,
this feature can be used consecutively any number of times, giving
the same stories every time.

.SPE
%A keyword expression may be continued over several lines.
Simply type a LINEFEED anywhere except in the middle of a word and
APE will type a carriage return and a colon (:) and wait for you to
type in more of the expression. A space is substituted for the linefeed.

.SPE
%Your current story list can be modified without typing
again the keywords you used to get it. If a keyword expression
starts with +, -, or *, the missing (first) operand is taken to be
your current story list. For example, if you have typed in "NIXON"
as your last keyword expression, you can type in "*VIETNAM" as your
next expression and you will get only stories that mention both NIXON
and VIETNAM.

.SPE
%When stories are typed out or written in a file, a row of
stars (*'s) is placed between stories. Note that corrections and
additions to a story are considered part of that story; thus they
will not be separated from it by a row of stars.

.SPE
%If you type control-O ([ESC] O on Stanford displays) during
typeout of a story, the typeout will be stopped (as usual), but will
start up again with the next story (if any).

.SPE ESCI:
%While APE is typing out and/or filing stories,
if you type [ESC] I (Stanford displays only),
or if you type control-C and then the system REEnter command, APE
will be back to asking for keywords, and your current story list
will not have been changed. (That is, you can get it back
by typing just carriage return; see paragraph {SPEC CURRENTLIST} above.)
Automatic command file reading is turned off when you do this.
Also, any file or spooler output going on is undone.

.SPE
%Upper and lower case characters are always equivalent.

.SPE
%If a keyword expression is preceded by a dollar sign ($),
APE will interpret that to mean that you wish to be notified
whenever a story comes in that fits the expression. (You will
still be told how many stories currently fit the expression.)
Whenever such a story does come in, a message will be sent
which you will get the next time you log in or use the RCV
program. The message will say something like this:
.BEGIN VERBATIM
FOUND (VIETNAM*PEACE)
IN STORY #321 1019pt 07-04
.END CONTINUE
where the time and date are those (pt=Pacific Time) that appear at the
beginning of the story.
Also, if you are logged in at the time the story comes in, the message
.BEGIN VERBATIM
*** AP STORY FOUND ***
.END CONTINUE
will be typed out on your console.
Notification is on the basis of your logged in programmer name;
however, programmer names 'GUE' and 'SYS' cannot use automatic notification
because there are many people using each of these names.
Also, notification requests cannot contain search strings (unrecognized
keywords) although this will probably change in the not too distant future.
Every notification request will expire eventually. The current
plan is to purge a request after it has existed for two months.
Whenever one of your requests expires, you will be sent a note like this:
.BEGIN VERBATIM
YOUR REQUEST (VIETNAM*PEACE)
EXPIRED BEFORE STORY #321 1019 07-04
.END
Automatic notification (AN) is intended to be used for two main purposes.
1)%If you are expecting an urgent story to come in at any moment, and you want
to be notified as soon as it comes in (assuming you are logged in), automatic
notification saves you the trouble of running APE every half hour to find out
if your story has come in. 2)%If you are expecting a story to come within
a couple of months, but you don't know exactly when, then AN saves you the
effort of running APE every day, if you wouldn't otherwise do so.

If you find you are being notified about the same kind of story several times
a day, and if the stories are not particularly urgent, then you will probably
find that the normal use of APE, possibly using a command file (see previous
section) will be more convenient. Also, the more AN requests there are, the more
work the continually running special AP programs have to do. However, you are
free to choose the method of using APE that best fits your purposes.

One final note on AN: When you get a hit from an AN request, the best way
to use APE to read the story is to type in the expression (possibly using a
command file) that got the hit.
Alternatively, you can type in the sequence number of the story found, but
this is liable to give you an extra story with the same sequence number.
You can combine these two methods and type something like (say) "#35*CHESS", if
CHESS was the AN request getting a hit on story #35.

.SPE
%If you type in a keyword expression that consists solely of a dollar
sign ($), then all notification requests you have in will be typed out
with their expiration dates.

.SPE
%If you enter the keyword expression "$$", then you will be permitted to
delete any of your notification requests.

.GROUP
.SEC NOTES
First, the news is kept in a fixed size file. This means that
old stories are continually being deleted to make room for new ones.
If this happens after you start APE, and if you attempt to read such
a deleted story, then you will get a message something like
"1%OF%THE STORIES WENT AWAY--SORRY".
.TURN OFF

Finally, news that comes in after you start APE cannot be retrieved.
If you want to update APE's data to include the latest stories, type
control-C and then the system START command. (When you do this your current
story list will be re-initialized to null.)